Linear Analog Codes: The Good and The Bad 



Kai Xie, and Jing Li (Tiffany) 
Electrical and Computer Engineering Department, Lehigh University, Bethlehem, PA 18015 

Email: {kax205, jingli}@ece.lehighe.edu 



Abstract — This paper studies tlie theory of linear analog 
error correction coding. Since classical concepts of minimum 
Hamming distance and minimum Euclidean distance fail in the 
analog context, a new metric, termed the "minimum (squared 
Euclidean) distance ratio," is defined. It is shown that linear 
analog codes that achieve the largest possible value of minimum 
distance ratio also achieve the smallest possible mean square 
error (MSE). Based on this achievability, a concept of "maximum 
distance ratio expansible (MDRE)" is established, in a spirit 
similar to maximum distance separable (MDS). Existing codes 
are evaluated, and it is shown that MDRE and MDS can be 
simultaneously achieved through careful design. 

I. Introduction 

In a typical digital communication system, analog source 
samples (such as sound and images) are first quantized and 
labeled to binary sequences, then encoded by a digital error 
correction code (DECC), and finally modulated by a digital 
modulator before being sent over the channel. However, quan- 
tization inevitably introduces irrecoverable granularity error, as 
well as significantly increases the data volume. Additionally, 
quantization also causes the issue of "most/more significant 
bits" vs "least/less significant bits", and would in general 
require judicious unequal error protection (UEP) to avoid 
wasteful protection of some bits while inadequate protection 
of others. An alternative to this cumbersome, although rather 
commonplace, digital approach is to completely rid of quanti- 
zation, directly encode analog data sequences to analog code- 
words, and transmit these analog symbols over the channel 
as in an oo-order amplitude shift keying (if real-valued) or 
quadratic amplitude modulation (if complex- valued). As such, 
a single analog error correction code (AECC) can replace the 
combination of quantization, DECC and digital modulation, 
and be free of quantization error floor 

The notion of analog error correction is not new. It ttaces 
back to the early 80's, when Marshall and Wolf independently 
introduced the concept [l]-[3]. It was termed real number 
coding in Marshall's work and analog coding in Wolf's work. 
Early work of AECC presents a natural outgrowth of linear 
digital codes, by extending conventional linear digital codes 
from the finite field to the real- or complex-valued field 
(symbols from a very large finite field can approximate real 
values). Hence, linear codes prevail the short literature of 
analog coding, just as they do in digital coding. There have 
also been proposals for nonlinear analog codes and some of 
them actually exhibited surprisingly good performances [8], 
[9]. In general, however, the research of analog coding is still 
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very limited, especially comparing to the high level of maturity 
of digital coding in both theoretical and practical contexts. 

It is apparent that analog codes generalize digital codes 
by relaxing the source space and the codeword space from 
discrete fields containing finite elements to continuous fields 
containing (uncountably) infinite elements [4]. However, it is 
less apparent what fundamental or subtle implications analog 
codes cast - especially in terms of coding theory and practical 
code design, compared to the usual practice of digital coding. 
Intuitively, Hamming distance, a concept of critical importance 
to digital codes, is much less indicative in analog codes, since 
two analog sequences can have a large Hamming distance 
but still be similar to each other (e.g. every symbol differs, 
but very minorly), or have a small Hamming distance but be 
far apart (e.g. only one symbol differs, but the difference is 
huge). However, Euclidean distance does not appear to be a 
good metric either As we will show, the minimum Euclidean 
distance of any analog code is infinitesimal, and hence analog 
codes do not have a guaranteed error correction capability as 
digital codes do. In the presence of additive white Gaussian 
noise (AWGN), every decoded frame of an analog code is 
bound to contain non-zero error/distortion, leading to an word 
error rate of always 1 . All of this departs from the conventional 
digital coding theory, and awaits to be illuminated. 

This paper presents a theoretic study of analog codes, and 
linear codes in particular. We develop several new concepts 
for analyzing and understanding linear analog codes, including 
the encoding power gain, the minimum (squared Euclidean) 
distance ratio and its achievable upper bound, and the mean 
square error (MSE) distortion and its achievable lower bound. 
We establish a concept of maximum distance ratio expansible 
(MDRE) for linear analog codes, which is close in spirit to 
maximum distance separable (MDS) in linear digital codes. 
We show that MDRE codes can achieve the best (i.e. largest) 
minimum distance ratio and the best (i.e. smallest) MSE 
distortion. In this, we show that all MDRE codes provide the 
same, best MSE distortion on AWGN channels. We identify 
analog codes that are MDRE (as well as MDS), and show that 
the criteria of MDRE and MDS, although evaluated against 
different distance metrics, need not conflict each other, but 
can be effectively unified in the same code design. We also 
evaluate existing linear analog codes and MDRE codes in 
particular. One important notion developed here is, unlike in 
digital coding where linear codes are sufficient to combat 
Gaussian noise, linear analog codes are actually rather weak 
and inadequate, and hence nonlinear transforms seem neces- 
sary, in order to provide performance that will be appreciated 
in practical applications. 



II. Preliminary 

In this paper, we use bold fonts, such as G and u, 
to denote matrices or vectors (column vectors by default), 
and use regular fonts, such as n and T to denote scalars. 
Superscript ^ denotes simple transpose of a vector or matrix, 
while superscript ^ denotes the Hermitian transpose. By 
default, all the analog codes have parameters (n, k), and maps 
a length-A; discrete-time complex-valued source sequence 



to a length-n discrete-time 



u 

complex-valued codeword v = {vo,v\,--- ,u„_i)^ e C". 
Since any linear code can be expressed in the form of a 
linear block code, we focus the discussion on linear block 
codes. Similar to digital codes, an analog linear block code 
is completely specified by its generator matrix, Gkxn, a 
rank-fc real-valued matrix satisfying v(u) = G^u. After the 
codeword v passes though a channel with additive noise w, 
the decoder receives r = v + w, and produces an estimate u 
of the original source vector u. 

Before we proceed, we first quickly overview the existing 
linear analog codes. The first, and one of the most important 
class, is discrete Fourier transform (DFT) codes, due to Mar- 
shall [1] and Wolf [2]. The generator matrix of an (n, k) DFT 
code is formed by extracting a set of k rows from the (normal- 
ized) DFT matrix '4' of order n, where each element '^i^k G ^ 
id defined as Vi = 0, 1, • • • , n — 1, A; = 0, 1, • • • , n — 1, 



2'ife 



(1) 



When the extracted rows follow certain structural formalism, 
the resultant complex DFT code can be viewed as an analog 
Bose-Chaudhuri-Hocquenghem (BCH) code and at the same 
time satisfies maximum distance separability [3]. In other 
words, there exist a subclass of DFT codes that are by nature 
analog Read-Solomon (RS) codes and hence optimal in the 
MDS sense. It has been shown that the traditional decoders of 
digital BCH codes, such as Feterson-Gorenstein-Zierler (PGZ) 
decoder, Berlekamp-Massey algorithm and Forney algorithm, 
are also applicable to analog BCH codes (and are useful when 
the channel is an erasure channel or a pulse channel). 

Another important class of analog codes, which happen to 
be also MDS, are discrete cosine transform (DCT) codes due 
to Wu and Shiu [5]. Similar to DFT codes, the generator matrix 
G of a DCT code comprises k selected rows from a DCT 
matrix H, where each element e H is defined as 



1 



cos 



n 
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k = l,2, 
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(2) 



Different from DFT codes, DCT codes are not analog BCH 
codes or even cyclic codes. However, since the parity check 
matrix preserves the properties of a Vandermonde matrix, 
DCT codes are nevertheless MDS. Additionally, a specific 
subclass of DCT codes can be expressed in a BCH-hke 
structure and decoded by a modified Berlekamp-Massey and 
Forney algorithm [5]. This BCH-like DCT structure was later 
generahzed to discrete sine transform (DST) codes by Rath 



and Guillemot [6], and a subspace-based decoder is proposed 
for general DCT and DST codes in [7]. 

We note that (normalized) DFT codes and DCT/DST codes 
are all unitary codes, i.e. codes whose generator matrix is 
formed by a selected set of rows from a square unitary matrix. 

III. Structure Properties: Distance Ratio & MDRE 

Clearly, generator matrices G and aG (a > 1) define 
essentially the same code, and the seemingly larger distance 
expansion of the latter is only the artifact of a larger consump- 
tion of transmission energy. To facilitate a fair comparison, we 
introduce the definition of encoding power gain. 

Definition 1: The encoding power gain F of a generator 
matrix G is defined as the ratio between the average codeword 
power and the average source vector power: 



(3) 



/P(u)u^urfu 

where P(u) is the probability density function (pdf) of the 
source vector u, and J f{v)dv represents the multiple inte- 
grals / / • • • / f{vo, ui, • • • , Vn-i)dvn-i ■ ■ ■ dvidvQ. An ana- 
log linear block code is said normalized, if F = 1. 

Theorem 1: Consider an analog linear block code with gen- 
erator matrix G. The source vector u consists of k elements uu 
each drawn from the same i.i.d (independent and identically 
distributed) source with distribution p. The encoding power 
gain is given by F = trace{GG^)/k. 

_ / P(u)v^vrfu _ /P(u)u^GG^urfu 

~ /P(u)u^^udu ~ /P(u)u^urfu ^ ' 

Since GG^ is Hermitian and positive definite, it is possible 
to perform a singular value decomposition, such that GG^ = 
A^DA, where A is a square unitary matrix and D is a 
real-valued diagonal matrix with positive diagonal elements 

{do,di,...,dk~i}. We have 

/ P(u)u^A^DAudu _ E^Jo(^JpK)l"^Prf"^) 



F = 



Y!IZI h{Ui)\'^^?du, k j p{uQ)\uQ\^dUQ 

k J p{uo)\uo\'^duo k 
trace{GG") 



(5) 



Corollary 2: Unitary codes have encoding gain T = k/k = 
1, and are therefore normalized. 

An error correction code provides error protection by ex- 
panding the distances among sequences. On AWGN channels, 
squared Euclidean distance becomes very relevant, since it 
constitutes the exponential part of the Gaussian distribution, 
and is closely related to the hkehhood test. The squared 
Euclidean distance of two codewords v and v' is given by 

4(v,v') = iiv-v'ip = x:rroM«.-«^p. 

Due to the geometric uniformity of hnear codes, the all- 
zero sequence is not only always a valid codeword, but can 
also act as a typical codeword in term of distance analysis. 
Just hke Hamming distance spectrum and Hamming weight 



spectrum are used interchangeably in linear digital codes, 
squared Euclidean distance spectrum and squared Euclidean 
weight spectrum are the same one in linear analog codes. 
The squared Euclidean weight of a codeword v is given by 
<(v) = 4(v,0). 

Theorem 3: The minimum squared EucUdean distance of an 
analog linear block code always approaches 0. 

Proof: It is sufficient to show that, for an arbitrarily small 
positive value e, there exists a codeword v = Gu whose 
squared Euclidean weight w%{-v) < e. Consider a source se- 
quence u = (mq, 0, 0)^ having only one non-zero element 
mq. The corresponding codeword v has weight: 

n— 1 n— 1 

^l(v) = I^^l' = IIG"""!!' = l^oP E l^^oP, (6) 

i=0 i=0 

where Qi^ is the element in the ith row and j'th column of G. 
Thus, if we select uq to be a real positive number satisfying 



< uo < 



(7) 



the corresponding codeword v(u) has a squared Euclidean 
weight w\ smaller than e. 

Since the minimum Euclidean distance/weight of analog 
linear codes can be arbitrarily small, it can no longer indicate 
the structural goodness of the code. Instead, we introduce a 
new metric, the distance ratio. 

Definition 2: Consider a pair of source sequences u and 
u' and their respective codewords v and v'. The squared 
Euclidean distance ratio, or simply, the distance ratio between 
them is defined as 



3^1 (u,u') 



4(v,v') 



c^l(u,u') ||u-u' 



/||2 ■ 



(8) 



The smallest distance ratio among all the source pairs is termed 
the minimum (squared Euclidean) distance ratio of the code. 
The average (squared Euclidean) distance ratio of the code is 
defined as 



iT)2 



P(u)3^|(u, 0)du. 



(9) 



Theorem 4: For all linear analog codes with encoding power 
gain r, their minimum distance ratio is upper bounded by F, 
and the upper bound is achieved by the generator matrix G 
such that all the k eigenvalues of GG^ are identical. 

Proof: It is sufficient to consider the distance ratio between 
an arbitrary non-zero codeword v(u) and the all-zero code- 
word. 

||u||"' u-"u 

Decompose GG^ to A^DA, where A is a unitary matrix 

and D is a diagonal matrix with positive diagonal elements 
{dQ,di, ...,dk-i}. Let dmin be the smallest of all the 
diagonal elements: dmin = minjrfo, rfi, rffe-i} > 0. We 
can simplify (10) to 



3?|(u,0) = 



u^A^DAu 



> dr, 



u^A^IAu 



u^A^Au 



u^u 



dr. 



(11) 



(12) 



where I is an identical matrix. The source vector u that 
achieves the equality in (11) is one that satisfies u' = Au = 
(0, 0, • • • , Wj, • • • , 0)^, where i is the index for dmin- 
Further, 



mm 



(3?|(u,0)) 



d„ 



< 



E,*=o di 



trace{GG") 



r. 



(13) 



(14) 



The equality in (13), i.e. the upper bound of the minimum 
distance ratio, is achieved when all the eigenvalues of GG^ 
are identical: di ~ d2 = ■ ■ ■ — rffe-i — dmin- 

Corollary 5: Given an (n, k) hnear analog code with gener- 
ator matrix G, its minimum distance ratio dmin is the smallest 
eigenvalue of the matrix GG^. 

Definition 3: Consider all the hnear analog codes with a 
fixed encoding power gain F. A code is called maximum 
distance ratio expansible or MDRE, if its minimum distance 
ratio achieves the upper bound F with equahty. 

Corollary 6: An (n, k) analog Unear block code with gen- 
erator matrix G is MDRE, if and only if all the k eigenvalues 
of GG^ are identical. 

Theorem 7: Analog unitary codes are MDRE. 

Proof: For an analog unitary code, we have GG^ = I. 
That is, GG^ has identical eigenvalues (i.e. 1) and the code 
is therefore MDRE. 

Since (normalized) DFT codes and DCT/DST codes are all 
unitary codes, they are also MDRE. What is less expected is 
that repetition codes are also MDRE. 

Theorem 8: Analog repetition codes are MDRE. 

Proof: Consider an analog repetition code that repeats the 
length-fc source vector t times. The generator matrix consists 
of r identity matrices of rank t each: G = [1^,1^, • • • ,Ife]. 
Since GG^ = tlk, all the eigenvalues are identical (i.e. t), 
and the code is therefore MDRE. 

Since MDRE codes do best in terms of distance expansion 
(given the same encoding power gain), it is reasonable to 
expect them to perform well - at least better than the rest of 
linear analog codes. However, the fact that analog repetition 
codes are MDRE, yet their digital counterparts are rather weak 
digital codes, suggests that even the best hnear analog codes 
may not be that good after all. In the next section, we analyze 
the performance of linear analog codes on AWGN channels, 
propose strategies to design codes that are both MDRE and 
MDS, and evaluate how good linear analog codes really are. 

IV. ML Decoding and Distortion 

We start by looking into optimal decoders and distortion 
metrics. Consider the noisy reception r at the decoder. The 
maximum-likelihood (ML) decoder for a general analog code 
produces u, where u = arg maxuP(r|u). 



For a linear analog code operating on an AWGN channel, 
the ML decoder transforms to an unconstrained convex opti- 
mization problem: 



u = are min llr — G'^ulP, 



(15) 



which can be solved analytically by expressing the objective 
function as a convex quadratic function 

||r - G"u\f = u^GG^u - 2r^G^u + r^r. (16) 



The ML decision is obtained as 

u= (GG^)-iGr. 



(17) 



We use mean square error (MSB) to evaluate the perfor- 
mance of an analog code on channels with additive noise w, 
where the MSE distortion is defined as 



A = y" (^P(u) J \\u-u\\^P{w)d-w^ du, 



(18) 



where ii is the decoder output for source u. For hnear 
analog codes, because of the geometric uniformity, instead of 
evaluating over all the possible source vectors u, the all-zero 
source vector can serve as the representative. Hence the MSE 
distortion can be simphfied to: 



A = y ||uo||^P(w)(iw, 



(19) 



where Uq is the decoder estimate for the all-zero codeword. 

Theorem 9: Consider an (n, k) linear analog code with 
encoder power gain V operating on an AWGN channel with 
noise w, where un Jsf(0,(7^). The mean square error 
distortion A after ML coding is lower bounded by 

(20) 



A > A„ 



r 



The lower bound is achieved by s§ = s\ = 
where {sq, si, ...Sk-i] are the set of singular values of G. 

Proof: Without loss of generality, assume that the all-zero 
codeword is transmitted. Substituting r = w and (17) in (19): 

A = y ||(GG^)-iGw||2p(w)dw 

(GG«)-iGw|p [](-^e-^)dw 
(GG^)-iGw||2e-%^dw 



(27ra2)"/2 

= / fw^B^Bw)- \- 

J V y'(27rc72)", 



/2' 



(21) 



where B = (GG~^)~^G. Since B^B can be decomposed 
into the product of A^DA, where A is a unitary matrix, and 
D is a diagonal matrix, we can simplifye (21) to 



A= / ( w^A^DAw) — ^-^( 



traceiTi) 
k 

trace{'D)a^ 



1 



(27ra2)»/2 



(22) 



The equality in 



(22) 



holds. 



/(w^w)- 



because 

- i.^2 



(i.e. the sum of the variance of Wi's). Note that 
B = (GG^)-^G is the psuedo-inverse of G. Let 
{so, •••Sfe-i} be the set of singular values of G, we have 



fc-i ^ 

trace(D) = trace{B"B) = ^ — 



(23) 



i=0 

fe-1 1 



To minimize A is then to minimize J2i=o i^' subject to 



Yj^^^ sf = kT, which leads to: 



'fc-i 



r. 



Hence we have 



A > 



ka^ 



(24) 



(25) 



Corollary 10: An MDRE code achieves the minimum 
bound of the MSE distortion on AWGN channels, and is 
therefore distortion optimal. 

V. Code Design 

We showed that unitary codes, of which DFT and DCT/DST 
codes are special cases, are MDRE, and hence promise decent 
performance on AWGN channels. Since DFT and DCT/DST 
codes are also MDS (in terms of Hamming distance), they 
will also perform well on erasure channels or pulse channels. 
Question then arises as how and how easy it is to design hnear 
analog codes that are both MDRE and MDS (and hopefully 
also simple). Below we provide a geometric view for hnear 
analog codes and propose useful design rules. 

A linear transform can de decomposed to a set of basic 
linear transformations: rotation, scaling, shearing, and reflec- 
tion. Consider an arbitrary generator matrix G, which can be 
singular value decomposed to G = ADB, where A and B 
are two square unitary matrices and D is a diagonal matrix 
whose diagonal elements are eigenvalues of G. This suggests 
that an arbitrary linear transform can be implemented in three 
steps: rotating via the rotation matrix A, followed by scaling 
via the scale matrix D, and followed by a second rotation via 
matrix B. Take a non-zero source vector u, and we evaluate 
how the rotate-scale-rotate process may affect its Hamming 
weight and Euchdean weight. (For linear codes, weight of a 
non-zero sequence translates to the distance between a pair 
of sequences.) The vector u initially spans a /c-dimensional 
subspace, which, when placed in an n-dimensional space, is 
like an n-dimensional vectors having [n — k) zeros in the last 
n—k dimensions. Since Hamming weight corresponds to the 
number of non-zero elements in the vector, scahng will not 
affect Hamming weight, but rotation will. Hence, as far as 
Hamming weight is concemed, one can safely assume that 
the scaling matrix is an identity matrix (no scahng on any 
dimension). What this implies in code design is that, for a 
given (n, k) analog code, be it MDS or not, it is always 
possible to find a unitary code that produces exactly the same 



Hamming weight spectrum. That is, a rotation matrix suffices 
to achieve the upper bound of the minimum Hamming weight. 

Now to put (squared) EucHdean weight in perspective, it is 
clear that rotation becomes irrelevant and scahng takes the 
determining role. From our previous analysis of maximum 
squared Euclidean distance ratio and the minimum MSE 
distortion, the best scaling should be one that is uniform across 
all the dimensions. This is why codes whose eigenvalues of 
the G^G are identical are MDRE and simultaneously achieve 
the best minimum distance ratio and the best MSE distortion. 

To conclude, the goals of optimizing Hamming distance 
and optimizing squared Euclidean distance do not conflict 
with each other in the context of linear analog codes. A 
good design can unify both metrics in one. For example, a 
carefully-selected matrix, such as that for an analog unitary 
code, achieves both MDRE and MDS bounds at one shot. 

A related issue concerns repetition codes, which, as we have 
shown, are MDRE, and hence exhibit the same, best MSE 
performance on AWGN channels as any other MDRE codes. 
This result is verified by our simulations in Fig. 1. The y-axis 
represent the MSE distortion in log-scale, i.e. log2(A), where 
A is defined in (18). The simulation curves clearly show that 
repetition codes and DCT codes perform exactly the same on 
AWGN channels (in terms of MSE distortion), both noticeably 
better than the other two randomly generated analog linear 
block codes of the same parameters. 

This result raises concerns on how good hnear analog codes 
really are, especially considering that digital repetition codes 
are rather poor codes amongst digital codes. An important 
finding we wish to report here is: while linear digital codes 
are sufficient in achieving channel capacity (as exemplified by 
turbo codes and LDPC codes), linear analog codes are not; 
and to really perform well, analog codes must go nonlinear. 
Specifically, we show that in Fig. 2 two recently reported 
nonlinear analog codes (Baker's map codes and CAT codes) 
[10] [9], both of which significantly outperform MDRE analog 
codes (the DCT codes). 

VI. Conclusion 

We have studied the theory of linear analog codes. We 
introduced the metric of minimum squared Euclidean distance 
ratio and mean square error distortion, established their re- 
spective upper bound and lower bound, and identified codes 
that simultaneously achieve these bounds. We also examined 
existing linear analog codes, and provided guidelines on 
designing codes that are both MDRE and MDS. While linear 
analog codes are (relatively) simple and analytically tractable, 
they are unfortunately weak. Unlike digital codes where linear 
codes are sufficient to combat AWGN, linear analog codes 
are inadequate and hence nonlinear analog mapping must be 
exploited for serious gains. 
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Fig. 2. Nonlinear analog codes can easily outperform linear analog codes. 
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